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SYSTEM AND METHOD FOR INTERACTIVE ADVERTISING 



CROSS-REFERENCE TO RELATED APPLICATIONS 

5 The present application claims the benefit of U.S. Provisional Patent 

Application Serial No, 60/224,736, filed August 10, 2000, entitled "USER 
INTERFACE TO TELEVISION BROADCASTS AND INTERNET CONTENT," which 
is incorporated herein by reference, 

10 BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates generally to communication systems and 
15 advertising, and in particular but not exclusively, relates to fulfillment of responses to 
advertisements sent over a communication system, such as an interactive video 
casting system. 

2. Background Information 

20 

An important business element in the production and distribution of 
television or radio programming is revenue received from manufacturers and service 
providers who pay to advertise their products. The ubiquitous television commercial 
is the manifestation of this enterprise. The survival of a television program is heavily 
25 dependent on the advertising revenue that can be realized from the television 
program. Advertisers in turn rely on the ability of the television program to draw 

1 
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viewers who then become potential purchasers of the advertised products. An 
effective commercial is one that captures the viewer's attention in a lasting manner 
and that ultimately results in the purchase of goods and/or services. 

In addition to advertisements presented as radio or television 
5 commercials, advertisements are also provided through other media. Examples 
include billboards, pre-recorded telemarketing calls, banner advertisements on 
Internet web sites, printed media such as "junk mail" or newspaper advertisements, 
and the like. These forms of advertisements typically invite the potential customer 
to dial a telephone number or mail a form in order to purchase the product or to 

10 obtain additional information. 

Advertisements do not always effectively result in a purchase for a 
variety of reasons. Among these reasons are that many advertisements provide 
insufficient interaction with the potential customer or mechanisms to make 
responding to the advertisements more convenient for the customer. A radio or 

15 television commercial for a local restaurant that provides home delivery serves as 
an illustration. In the commercial, the local restaurant may advertise a type of pizza, 
followed by a telephone number that the potential customer can call to place an 
order. If the potential customer is interested in having pizza delivered to his home 
from that restaurant, the customer has to stop what he is doing, write down or try to 

20 remember the telephone number (or address) of the restaurant, dial the telephone 
number, and then perhaps eventually place an order with an operator at the 
restaurant who answers the telephone call (assuming that the customer is not kept 
on hold while other orders are taken). Similar types of multiple tasks need to be 
completed by potential customers if these customers wish to respond to other forms 

25 of advertisements, such as billboards, printed media, etc. In short, these forms of 
advertisements involve a lengthy and cumbersome process that needs to be 
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performed by the potential customer. Such advertisements make it inconvenient to 
potential customers to respond to advertisements, thereby ultimately reducing the 
customers' inclination to begin or complete a purchase. 

Accordingly, improvements are needed in the advertisement of 
5 products and in the fulfillment of customer responses to the advertisements. 



3 



Attorney Docket: 005217.P022 

SUMMARY OF THE INVENTION 

The above-described problems relating to advertisements are addressed by the 
present Invention. The present invention comprises a system and method for 

5 interactive advertising. In accordance with an embodiment of the Invention, an 
advertising service presents an interactive advertisement to a customer, such as via 
a television commercial. If the customer Indicates an interest in the advertised 
product by responding to the advertisement, then the advertising service or other 
mechanism detects the customer's response, identifies the customer, correlates the 

10 customer to a merchant, and then automatically connects the customer to the 
merchant (or triggers connection of the merchant to the customer). Thereafter, the 
merchant can fulfill the customer's response in a personalized and interactive 
manner. An aspect of the invention provides a method to detect a response, to an 
interactive advertisement, sent from a client terminal of a customer. The method 

15 identifies customer information from the detected response. The identified customer 
information is correlated to merchant Information to Identify a specific merchant, and 
notification of the identified specific merchant is triggered to allow communication 
between the customer and the specific merchant. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Non-limiting and non-exhaustive embodinnents of the present invention 
are described with reference to the following figures, wherein like reference 
5 numerals refer to like parts throughout the various views unless otherwise specified. 

Figure 1 shows an example of an interactive video casting system that 
can implement interactive advertising in accordance with an embodiment of the 
invention. 

Figure 2 shows an example of an interactive video casting system that 
10 can implement interactive advertising in accordance with an embodiment of the 
invention. 

Figure 3 shows an example of an interactive video casting system that 
can implement interactive advertising in accordance with an embodiment of the 
invention. 

15 Figure 4 depicts an interactive advertisement that can be presented to 

a customer via the systems of Figures 1-3 in accordance with an embodiment of the 
invention. 

Figure 5 is a flow diagram depicting a sequence of events in 
connection with providing, and fulfillment of responses to, the interactive 
20 advertisement of Figure 4. 

Figure 6 is a flow diagram depicting a video and audio capture and 
communication technique that can be used to fulfill a customer's response to the 
interactive advertisement of Figure 4. 
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DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS 



Embodiments of a system and method for interactive advertisement 
are described herein. In the following description, numerous specific details are 
5 provided, such as the description of system components in Figures 1-3, to provide a 
thorough understanding of embodiments of the invention. One skilled in the 
relevant art will recognize, however, that the invention can be practiced without one 
or more of the specific details, or with other methods, components, materials, etc. 
In other instances, well-known structures, materials, or operations are not shown or 

10 described in detail to avoid obscuring aspects of the invention. 

Reference throughout this specification to "one embodiment" or "an 
embodiment" means that a particular feature, structure, or characteristic described 
in connection with the embodiment is included in at least one embodiment of the 
present invention. Thus, the appearances of the phrases "in one embodiment" or "in 

15 an embodiment" in various places throughout this specification are not necessarily 
all referring to the same embodiment. Furthermore, the particular features, 
structures, or characteristics may be combined in any suitable manner in one or 
more embodiments. 

As an overview, an embodiment of the invention provides a form of 

20 advertising (and the resulting fulfillment of customer responses to the 
advertisement) that can be thought of as "interactive advertising." For an 
advertisement for a national or well-known product {e.g., goods or services), for 
instance, an embodiment of the invention correlates a potential customer's response 
to the advertisement to a specific local merchant (such as a local retailer, distributor, 

25 or service provider) for fulfillment of the customer's response. Such fulfillment can 
include the service of the customer's order or the providing of additional information. 



6 



Attorney Docket: 005217.P022 

In some embodiments, the merchant that fulfills the customer's response need not 
necessarily be local, and instead can be located anywhere and still fulfill the 
customer's response. 

In an embodiment, an advertising service presents an interactive 
5 advertisement to a customer, such as via a television commercial. If the customer 
indicates an interest in the advertised product by responding to the advertisement, 
then the advertising service or other mechanism detects the customer's response, 
identifies the customer, correlates the customer to a merchant, and then 
automatically connects the customer to the merchant (or triggers connection of the 

10 merchant to the customer). Thereafter, the merchant can fulfill the customer's 
response in a personalized and interactive manner. 

For simplicity of explanation and for illustrative purposes, an 
embodiment of the invention will be described herein as being implemented in an 
interactive video casting system, such as an interactive television system. It is 

15 understood that other embodiments of the invention can be implemented in other 
advertising media, such as computer networks, radio, telephone, or other media or 
combinations thereof that are capable of interactive advertising as disclosed or 
taught herein. Where appropriate, descriptions of such alternative or additional 
implementations are provided herein. 

20 Figure 1 shows an example arrangement of an interactive video 

casting system 100 that can implement interactive advertising in accordance with an 
embodiment of the invention. A production company 104 produces programming 
content for transmission to viewers. The transmission is sent over an uplink channel 
to a satellite 102. The satellite 102 then transmits the programming content over a 

25 downlink channel to a local studio 106. The local studio 106 can insert additional 
programming {e.g,, regional programming) and/or advertisements as needed into 
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the programming content. In this regard, the local studio 106 can act as an 
"advertising service" for interested or subscribing merchants. 

The content with the insertions is then transmitted from the local studio 
106 to a cable service provider 108. In an embodiment, the television program may 
5 be downloaded to a receiving station, such as a head-end (H/E) of the cable service 
provider 108, rather than or in addition to the local studio 106. A reverse channel 
from the cable service provider 1 08 to the local studio 1 06 is provided so that the 
local studio 106 can insert additional programming content or advertisements, and 
feed the television signal back to the cable service provider 108. The cable service 

10 provider 108 then delivers the television signal over a cable network 134 to cable 
subscribers. In addition or alternatively to the local studio 106, the cable service 
provider 108 can also act as an advertising service that is also capable to insert 
advertisements (and/or other information related to the advertisements, such as 
triggers, data to identify the merchant, time of broadcast of the advertisement, 

15 address of the merchant, etc.) into the television signal. 

The cable network 134 is provided by the cable service provider 108 to 
distribute the programming content to cable subscribers. A set top box (STB) 152, 
located on the premises of a cable television subscriber {e.g., a user or customer), 
receives the programming content or television signal, and delivers the television 

20 signal to the subscriber's television set 154. Alternatively or in addition, the 
television signal can be broadcast over a wireless medium and received by a 
traditional aerial antenna or by a satellite dish, and then delivered to the set top box 
152. Alternatively or additionally, features and functionality of the set top box 152 
may be integrated into a type of advanced television or other display device. 

25 Moreover, embodiments of the invention can use other types of 

broadcast media, including but not limited to, digital cable systems, satellite, very- 
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high-data-rate digital subscriber line (VDSL), web casts, etc. The features provided 
by the television set 154 can also be provisioned, in one embodiment, by a personal 
computer (PC) suitably configured with an adapter to convert television signals into 
a digitized format, and then to deliver the television signals to the video portion of 
5 the computer for display. It is noted that the invention is not limited to any one 
configuration of display hardware, as embodiments of the invention will work equally 
well using alternative reception and display arrangements. 

In accordance with an embodiment of the invention, a connection to a 
communication network is provided for the cable subscriber. In one embodiment, 

10 the connection can be made via a cable modem 156 over a bi-directional 
communication link 155 to a cable modem termination system within the cable 
provider's 108 equipment. The connection continues to a data communication 
network, such as the Internet, by way of a public switched network (PSTN) 132, 
The PSTN network 132 is provided herein as an example, and it is understood that 

15 other types of networks may be used for connectivity to the Internet. A cable 
modem arrangement can be used because of its high bandwidth capability. In 
situations where some cable companies are not equipped to provide cable modem 
service to their customers, various other arrangements can be made. For example, 
a conventional modem connection can be used to access the Internet over a 

20 telephone line. 

As another example, Internet access can be gained over a DSL 
connection or an integrated services digital network (ISDN) connection using a 
telephone line. Wireless systems are also available for providing Internet access. 
In one embodiment, downstream data transmission can occur via cable or satellite, 

25 and upstream data transmission can occur via a telephone line. It is also to be 
appreciated that the customer may have other communication devices, such as a 

9 
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telephone or a separate PC, which are capable of providing communications with a 
merchant 122 via the PSTN 132 or other data communication network, 
independently or in conjunction with a television signal that is received by the set top 
box 152. 

5 It is noted that the Internet is chosen as an example of a data 

communication network because it is a well-established network, and connectivity to 
the Internet is easily made. However, it is noted that a global communication 
network, such as the Internet, is not required to practice other embodiments of the 
invention. A locally provided and maintained communication network may be used 

10 in an embodiment. 

Continuing with Figure 1, the set top box 152 can include a transceiver 
157, such as an infrared (IR) or radio frequency (RF) transceiver, that can exchange 
signals with a remote control unit 158 or other user input device. The set top box 
152 can be a component that is separate from the television set 154 as shown in 

15 Figure 1, or its features can be built into circuitry of the television set 154 [e.g., an 
interactive television set). The set top box 152 enables a viewer to select a 
television program to view and then delivers the television program to the television 
set 154. 

In an embodiment, the set top box 152 can be provided with an 
20 identification number or address that uniquely identifies the set top box 152 within 
the system 100 from other set top boxes. Such an identification number or address 
can be installed by a manufacturer {e.g., "hard-coded" into the set top box 152 at the 
time of manufacture, for example), or provided/assigned when the customer 
subscribes to interactive television services from the cable service provider 108. An 
25 address for the set top box 152 can include an Internet Protocol (IP) address, 
uniform resource locator (URL) address, or other address. As will be described later 
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below, this identification information can be used in an embodiment of the invention 
to allow the advertising service or the merchant 122 to identify the customer when 
the customer sends a response to an advertisement from the set top box 152. 

A storage unit 162 can also be coupled to or be a part of the set top 
5 box 152. The storage unit 162 can comprise a machine-readable storage medium 
such as a cache, buffer, memory, diskette, compact disk, tape, or the like and their 
associated hardware, in one embodiment. In another embodiment the storage unit 
162 can include a video cassette recorder (VCR). In another embodiment, the 
storage unit can include a hard disk such as a digital or personal video recorder 

10 (DVR or PVR). 

As noted above, the local studio 106 can insert additional 
programming into the received transmission, for example, to provide cable content 
that includes locally provided channels. The programming is then distributed to 
customers over the cable network 134. In addition to local program insertion, the 

15 local studio 106, cable service provider 108, or other party can insert advertising 
content. Product supplemental information relating to the advertising for 
participating merchants 122 can also be inserted. Product supplemental information 
can include information relating to the goods or services being advertised in the 
commercial. In addition to goods and services, coupons and other information 

20 services can be made available to the viewer, which in one embodiment can be 
obtained via the merchant's 122 web site 124 on the Internet. Triggers, such as 
Advanced Television Enhancement Forum (ATVEF) triggers, which are related to 
the web site 124 and/or to its contents, can be continuously updated as the 
television broadcast is being received. The web site 124 can be provided from a 

25 server, which can also send information to the set top box 152 in connection with 
fulfilling customer responses to advertisements. Such information sent from the 
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server to the set top box 152 or other terminal of the customer can include, but not 
be limited to, an email, an order form, requested product information, confirmation of 
orders, or other information in electronic format capable of being sent to the 
customer. 

5 In an embodiment of the invention, the merchant 122 may 

subscribe/register or othenwise make itself known to the local studio 106, the cable 
service provider 108, or other advertising service. This registration allows the 
advertising service to provide advertisements on behalf of the merchant 122 and to 
subsequently correlate customer responses to the interactive advertisements to the 

10 merchant 122. This registration of the merchant 122 is represented in Figure 1 by a 
dashed line from the merchant 122 to the local studio 106. The dashed line also 
represents that the local studio 106 (or other advertising service) can notify the 
merchant 122 of customer responses to interactive advertisements, with the 
notification capable of being sent via the PSTN 132 or other communication 

15 network/medium. 

In accordance with an embodiment of the invention, a participating 
merchant list 153 may be maintained. As the name implies, this list permits 
participating merchants 122 to provide their product supplemental information to the 
viewer. The participating merchant list 153 may be provided to and stored in the set 

20 top box 152. Alternately or additionally, the participating merchant list 153 may be 
stored at a head-end or other system of the cable service provider 108, or at a third 
party system. In this embodiment, the participating merchant list 153 may operate 
as a "white" list which allows transmission of triggers from authorized merchants and 
filters out other triggers. In another embodiment, a "blocked" or "black" list may be 

25 maintained at the set top box 152 or elsewhere. Such a blocked list filters out 
undesirable triggers and may be created and/or maintained by the cable service 
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provider 108. Alternately or additionally, such a blocked list may be edited by an 
end user. 

Various techniques for carrying the product supplemental information 
can be used. For example, triggering, announcement, or resource information can 
5 be included and sent using the ATVEF standard, in a manner known by those 
skilled in the art. For instance, a uniform resource locator (URL) address can be 
embedded in the broadcast stream. Other standards that may be used include 
triggering mechanisms from Wink and WorldGate. Another technique is to embed 
code or a script in the stream that runs on the client {e.g., at the set top box 152) to 

10 provide the information and/or purchase experience. 

As noted above, the triggers, resources, or announcements can be 
inserted by the originating broadcaster 104, by a local studio 106, by the cable 
service provider 108, or by another party. Figure 2 shows another example of an 
interactive video casting system 200 and illustrates another point of insertion of the 

15 product supplemental information. Here, a merchant 222, operating a web site 224, 
is located such that commercial insertion is made prior to the uplink transmission to 
the satellite 102. The merchant 222 can also be subscribed similarly as the 
merchant 122 and be capable of fulfilling customer responses that are directed to it. 

As noted above, Internet access is not necessary to practice the 

20 invention. A locally provided network may be within the scope of the invention as 
claimed. The cable provider 108 can supply the foregoing features, for example, by 
providing a web site or "walled garden" that is accessed by its subscribers. In such 
a case, the cable provider 108 serves as an intermediary and submits the purchases 
to the actual merchants 122 or 222. 

25 Figure 3 shows another example of an interactive video casting system 

300 that can implement interactive advertising in accordance with an embodiment of 
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the invention. The system 300 can be similar to or combined with the systems 100 
and 200 shown in Figures 1 and 2, respectively. In accordance with an embodiment 
of the present invention, the system 300 can be integrated with a cable television 
distribution system. The system 300 includes an Internet 302, a plurality of content 
5 sources 304, a plurality of distribution centers (depicted as the head-ends or H/Es 
306), and a plurality of client terminals 308 (depicted as set top boxes). In addition, 
a content source 304 is depicted as receiving data from data feeds 312, 
advertisement servers 314, image sources 316, and streaming video sources 318. 

The plurality of content sources 304 is coupled to the Internet 302. 

10 For example, a content source 304 may comprise a web site portal such as 
Go2Net.com, or a news web site such as CNN.com, or other types of sources. 
Each content source 304 may have various data feeds 312, servers 314, and 
sources 316/318 coupled to it. In an embodiment, the content source 304 can 
include an advertising service, and can be embodied in parties such as a local 

15 broadcaster or a multiple system operator (MSO), similar to the local studio 106 and 
cable service provider 108, respectively, of Figures 1-2. 

News or stock quote feeds 312 may be fed into the content source 
304. Servers 314 may provide advertisements for insertion into multimedia content 
delivered by the content source 304. Sources 316/318 may provide images 316, 

20 streaming video 318, and other content to the content source 304. Various other 
feeds, servers and sources may also be coupled to the content source 304 of Figure 
3, or coupled to the production company 104, cable network 134, web sites 124 and 
224, or to other components of the systems shown in Figures 1 and 2. 

The Internet 302 comprises a network of networks and is well known in 

25 the art. Communications over the Internet 302 can be accomplished using standard 
protocols such as transmission control protocol/internet protocol (TCP/IP), hypertext 
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transfer protocol (HTTP), file transfer protocol (FTP), or other protocols. The 
Internet 302 is coupled to the plurality of distribution centers 306, and each 
distribution center 306 is in turn coupled to a plurality of client ternninals 308, which 
nnay comprise a set top box, a PC, an interactive television set, or another type of 
5 communication device or display device. 

In alternative or in addition to the Internet 302 being used to distribute 
multimedia content from the content sources 304 to the distribution centers 306, 
communications channels or networks 320 apart from the Internet 302 may couple 
one or more content sources 304 to one or more distribution centers 306. One 

10 example of such an alternate path for communications is illustrated by a first dashed 
line 320 in Figure 3. Alternately or additionally, peering connections may exist 
between distribution centers 306. One example of such peering is illustrated by a 
second dashed line 322 in Figure 3. Other configurations are also possible and are 
included within the scope of the present invention. 

15 Caches 310 may be provided at (or coupled to) the distribution centers 

306. Such caches 310 may be used to increase the performance in the delivery of 
multimedia content to the client terminals 308. For example, larger files for video 
and other high bandwidth content may be stored in such caches 310, which may be 
closer to the client terminals 308 than to the content sources 304. In addition, 

20 reliability and guaranteed bandwidth may be provided because the Internet 302 is 
not in-between such caches 310 and the client terminals 308. 

The distribution centers 306 {e.g., head-ends) may include a server, in 
one embodiment, to direct a customer's response to an advertisement to the web 
site 124 of the merchant 122, for example. The distribution center 306 may also 

25 include, or be communicatively coupled to, a database (having merchant, customer, 
or advertising/broadcast information stored therein), such that when a response 
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from the client terminal 308 is received, a database lookup can be performed by the 
server or software to identify the client terminal 308 that responded, to correlate the 
response with the advertisement that it responded to and with the corresponding 
merchant, and to trigger notification of the corresponding merchant to contact the 
5 customer or client terminal 308 to fulfill the response. 

The server may also include software to redirect a browser of the 
customer's client terminal 308 to the merchant's web site, for example, if the 
merchant is a type that fulfills responses via its web site. In this manner, the 
merchant can identify the customer after redirection and begin fulfillment of the 

10 customer's response. In another embodiment, the distribution center 306 can 
include or be coupled to a communication device/equipment to notify the merchant 
after the customer's response has been detected and after the customer has been 
identified. Such equipment can include telephone interfaces to notify the merchant 
via an automated telephone message or a facsimile, an email interface to provide 

15 an email message to the merchant, a wireless device interface to provide a 
notification to the merchant via a wireless medium, etc. 

In accordance with one embodiment of the invention, different or 
multiple portals may be used to access the information provided through the 
interactive video casting systems of Figures 1-3, based on the type of client terminal 

20 being used by the end user. That is, for example, a television portal may be 
provided for an end user that uses the television set 154 to access the information. 
A PC portal may be provided for an end user that uses a PC to access the 
information. Portals can be provided for end users that use cellular telephones, 
PDAs, audio devices, etc. to access the interactive video casting systems of Figures 

25 1-3. 



Attorney Docket: 005217.F022 

Such portals may be provided in several possible ways. In one 
embodiment, the client terminal {e.g., the end user's display device or audio device) 
can be suitably configured with an adapter that includes hardware and software. 
The adapter converts the television signals, the Internet or web page content, or 
5 other information provided from the interactive video casting system into a digitized 
format or other format that is compatible with the operational features of the client 
terminal. 

In another embodiment, the cable provider 108 can deliver signals 
having different formats to the various client terminals, with the client terminals not 

10 necessarily having special adapters. Therefore, as an example, the cable provider 
108 or other party can generate/deliver information {e.g., television programming, 
web page content, etc.) having a format that is compatible for end users that receive 
the information via the television set 154. The cable provider 108 or other party can 
also generate/deliver the same information (e.g., simultaneously with the television 

15 portal on the same communication link, separately on a different communication 
link, on-demand independent of the television portal, etc.) using a format that is 
compatible with end users that receive the information via PCs, PDAs, cellular 
telephones, etc. Thus, the term Interactive video casting system" is used to 
describe generally a system that can deliver video information over any network and 

20 any network-compatible device by broadcasting, multicasting, or unicasting. An 
"interactive television system" is one type of or one means of access to an 
"interactive video casting system." 

Referring next to Figure 4, shown at 402 is example of an interactive 
advertisement that can be presented to a potential customer via the systems of 

25 Figures 1-3 in accordance with an embodiment of the invention. In the example 
shown in Figure 4, the interactive advertisement 402 is a commercial for airline 
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tickets, and it is to be appreciated that other types of broadcast segments may be 
displayed, such as public service announcements, previews of upcoming 
programming, "infommerciais," or other programming that can be displayed as part 
of an interactive video casting transmission. Furthermore, the broadcast segments 
5 or the interactive video casting transmission need not be live, and may be pre- 
recorded, played from a VCR, buffered prior to display, etc. or any suitable 
combination of these methods. 

The interactive advertisement 402 may have an audio or visual (or 
both) indicator 404 to indicate the availability of enhanced content {e.g., product 

10 supplemental information), which the viewer can access using a viewer input device. 
In the example shown in Figure 4, the indicator 404 is a "Buy" button that invites the 
viewer to buy airline tickets. The indicator 404 may be an "Info" button that requests 
the merchant to provide information about the tickets, there may be multiple 
indicators 404, or one or more indicators may be used for a combination of 

15 purposes {e.g., buying, requesting information, turning enhanced content on/off, 
activating a menu, etc.), according to various embodiments. 

Activation of the indicator 404, by pressing a buy button 174 or an 
information button 172 of the remote control unit 158 shown in Figures 1-2, 
according to one embodiment, initiates the customer's response to the interactive 

20 advertisement 402 by sending a command to the set top box 152 to request 
fulfillment of the response. Thereafter, the activation of the indicator 404 is detected 
by the advertising service, the customer is identified, customer information is 
correlated with the particular interactive advertisement 402 and/or with its 
corresponding merchant, the corresponding merchant (such as a local affiliate 

25 merchant of the airline company that sells the advertised tickets) is notified by the 
advertising service, and then the merchant communicates with the customer to fulfill 
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the response, all of which can be done in a manner according to various 
embodiments that will be described as follows, using the systems shown in Figures 
1 -3 as examples. 

Referring next to Figure 5, shown generally at 502 is a flow diagram 
5 depicting a sequence of events in connection with providing, and fulfilling a 
customer response to, the interactive advertisement 402 of Figure 4 or other 
interactive advertisement. Portions of the flow diagram 502 can be embodied in 
software or other machine-readable instructions stored in one or more machine- 
readable media. The machine-readable media can be located in the set top box 
10 152 or other local storage unit, in a head-end 306, in one or more servers in the 
systems shown in Figures 1-3, or in other locations or any suitable combination of 
these locations. 

Beginning at a block 504, the merchant is registered with an 
advertising service, so that the advertising service can provide interactive 

15 advertisements on behalf of the merchant as part of a broadcast. This registration 
of the merchant, such as the merchant 122, provides the advertising service with the 
merchant's contact information (e.g., address, telephone number, facsimile number, 
email address, URL of the web site 124, etc.), the preferred method of contact by 
the advertising service and/or by the customer, preferred categories of customers 

20 (such as customers in a particular geographic area or customers who are 
determined to be frequent shoppers for a particular product, which can be 
determined by user profiling techniques), or other information usable by the 
advertising service in connection with providing advertisements to customers and 
subsequently directing the customer to the appropriate merchant for fulfillment of 

25 responses to the advertisements. The registration of the merchant can also further 
include, in one embodiment, enablement of a feature to redirect the customer to the 
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merchant 122 when the Buy button indicator 404 is clicked. The merchant 
information may be stored in a database, such as in the cache 310 or other storage 
location coupled to the distribution center 306 of Figure 3. 

Next at a block 506, the interactive advertisement 402 is transmitted to 
5 a client terminal of the customer, such as to the set top box 152 in one embodiment. 
In another embodiment, an interactive advertisement may be transmitted via radio, 
via telephone, via a web site, or via another medium capable to implement 
interactive advertising. 

When the customer that has seen or heard the interactive 

10 advertisement 402 decides to order the advertised product, the customer indicates 
his intention to order or his interest in the advertised product. Such an indication 
can include, in one embodiment, communication through a tactile input device (such 
as a keyboard, telephone keypad, or the remote control unit 158). For instance, the 
user can use the remote control unit 158 to click on the Buy button indicator 404 

15 shown in Figure 4. If the customer is viewing the interactive advertisement 402 on a 
PC, then a mouse or other pointing device may be used to activate the indicator 404 
or other element of the interactive advertisement indicative of a customer response. 
The set top box 152 (or the PC) then communicates the customer's response to a 
head-end or distribution center 306, a server, or other network component of the 

20 advertising service. At a block 508, the customer's response is detected, which in 
an embodiment involves reception of the uplink transmission from the set top box 
152. 

Next at a block 510, the customer is identified and correlated with the 
interactive advertisement 402 and/or its corresponding merchant. In an 
25 embodiment, the customer can be identified by reading/detecting the contents of the 
uplink transmission from the set top box 152. That is, the uplink transmission can 
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include the unique identification number of the set top box 152 or its unique network 
address, and therefore can be used to identify the customer by matching the 
detected unique identification number with customer and identification information 
stored in a database. Once identified, additional customer information can be 
5 obtained from the database, such as name, telephone number, street address, 
email address, etc. 

Using a database lookup technique in one embodiment, the identified 
customer can then be correlated to the particular interactive advertisement 402 at 
the block 510, and/or a characteristic associated with the particular interactive 

10 advertisement 402 can be correlated to a specific merchant 122. For instance, an 
embodiment can correlate the identified customer to the particular interactive 
advertisement 402 that was clicked, the television channel in which the interactive 
advertisement 402 was displayed, the time in which the interactive advertisement 
402 was displayed and/or when the indicator 404 was clicked, the specific merchant 

15 122 associated with the interactive advertisement 402, etc. Identification of the 
specific merchant 122 appropriate to service the customer's response can involve a 
database lookup of local merchants from a list of merchants, based on the 
customer's geographic location, based on the type of response the customer sent, 
or other criteria. 

20 Once correlated, the merchant 122 is notified by the advertising 

service (or other mechanism) at a block 512 to contact the customer. In an 
embodiment, the merchant 122 is notified implicitly by redirection of the customer's 
browser or set top box 152 to the merchant's 122 web site 124, for example. This 
redirection may be performed by a server in the distribution center 306 in one 

25 embodiment, and results in a connection of the merchant 122 to the customer at a 
block 514. This embodiment can be implemented via use of triggers that are 
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embedded with the television transmission having the interactive advertisement 402. 
When the indicator 404 is clicked, the set top box 152 is redirected to the URL 
address of the merchant's web site 124, with the URL addresses capable of being 
obtained from the triggers or from other embedded information that is sent along 
5 with the interactive advertisement 402. Once at the web site 124, the customer can 
communicate with the merchant 122 by filling out an electronic form or by providing 
other input (such as by clicking hyperlinks) to get the response fulfilled. 

In another embodiment, the merchant 124 is notified explicitly by the 
advertising service (or other mechanism) at the block 512, and then the merchant 
10 122 directly and actively connects to the customer at the block 514. Explicit 
notification of the merchant 122 at the block 512 can be implemented in several 
ways once the customer and the merchant 1 22 have been correlated at the block 
510. 

In one embodiment, the merchant 122 may be notified by an 
15 automatically generated email (sent from a server at the distribution center 306 or 
other location, for instance) that can include the customer's contact information and 
description of the customer request. In another embodiment, the merchant 122 may 
be notified by a telephone call from the advertising service (such as from a human 
operator or via an automated message), which may be sent via the PSTN 132 and 
20 which provides the customer information needed by the merchant 122. It is 
understood that other forms of notification can be used as well, such as facsimile, 
page, screen alert, instant message, etc. 

In accordance with one embodiment, a template can be created by the 
advertising service for the merchant 122. Such a template can contain fields for the 
25 customer's contact information and description of requested items. When the 
merchant 122 is to be notified, a template is completed for the specific customer (by 
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electronically extracting customer and merchant information from a database), and 
then the completed template is sent to the merchant 122, The completed template 
can be provided to the merchant 122 as a display screen "pop-up" window or other 
visual presentation, as a facsimile, as an automated telephone message, etc. In an 
5 embodiment, since the merchant 122 is already provided with the customer's 
contact information in the completed template, the merchant can simply press a 
button to automatically connect to the customer. Pressing of the button can trigger 
a speed dial telephone call to the customer, an email to the customer, initiation of a 
video communication with the customer, etc. in an embodiment, the merchant 122 

10 can subscribe to have this display screen "pop-up" service in which it is provided 
completed templates. 

Connection between the merchant 122 and the customer can be 
implemented in several ways at the block 514, alternatively or in addition to the 
techniques described above. According to various embodiments, the merchant 122 

15 can connect to the customer and/or the customer can be connected to the merchant 
122. In one embodiment, the customer can be connected to the merchant 122 (or 
vice versa) by connecting the customer's telephone (such as via the PSTN 132, a 
computer-integrated telephone, a telephone provided as part of the customer's 
interactive television service, or some other type of voice communication service) to 

20 the merchant's 122 telephone number. In another embodiment, the customer can 
be connected to the merchant 122 (or vice versa) by linking the customer's 
computer to the computer or other terminal of the merchant 122 through a network, 
such as the Internet 302 of Figure 3. 

As described numerous times herein, communication between the 

25 merchant 122 and the customer at the block 516 to fulfill the customer's response 
can utilize many different types of communication media/formats, including a live 
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telephone call between the customer and a service representative of the merchant 
122, In accordance with an embodiment of the invention, communication between 
the customer and the merchant 122 at the block 516 may be conducted by way of 
video communication, sometimes referred to as "video conferencing." Such video 
5 communication can implement conventional Internet video conferencing techniques 
in one embodiment. Another embodiment can use video/audio capture and 
communication techniques similar to those described in U.S. Patent Application 
Serial No. 09/698,298, entitled "SYSTEMS, METHODS, AND DEVICES FOR 
VIDEO AND AUDIO CAPTURE AND COMMUNICATION," filed October 27, 2000, 
10 which claims priority based on U.S. Provisional Application Serial No, 60/237,013, 
entitled "SYSTEMS, METHODS, AND DEVICES FOR VIDEO AND AUDIO 
CAPTURE AND COMMUNICATIONS," filed September 29, 2000, both of which are 
owned by the same assignee as the present application and which are incorporated 
herein by reference. 

15 An embodiment of such a video and audio capture and communication 

technique that can be used during fulfillment of a customer's response to any 
interactive advertisement, such as the interactive advertisement 402 of Figure 4, is 
shown in a flow diagram 600 in Figure 6. The technique depicted in the flow 
diagram 600 can be implemented using suitably located web cameras and 

20 associated microphones or other audio capture devices. Such a 
camera/microphone may be mounted on the set top box 152, on the television set 
154, integrated in the remote control unit 158, or located elsewhere proximate to the 
customer. For simplicity of explanation, the flow diagram 600 will be described 
hereinafter in the context of the customer as the party that initiates a video 

25 conference communication with the merchant 122, and it is clear that the described 
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procedures can be applied in the context of the merchant 122 as the party that 
initiates the video conference connmunication with the customer. 

The flow diagram 600 begins at a block 602 when the customer uses 
an input device, such as the remote control unit 158 in conjunction with the set top 
5 box 152, to select a destination {e.g., the merchant 122) to receive a video/audio 
transmission. The selection may be performed by entering an identification or 
network address of the merchant 122, or by clicking on the indicator 404 using the 
remote control unit 158 in one embodiment. If the identification or network address 
of the merchant 122 is not known, for example, the set top box 152 may access a 

10 name server or directory (not shown) to retrieve a corresponding identification or 
address of the merchant 122. In one embodiment, the set top box 152 may contain 
a local directory of addresses, such as the participating merchant list 153 or other 
list, to which the user frequently sends video/audio transmissions. 

Once the set top box 152 has a valid address, it sends a request 

15 across the network to the merchant 122. The precise format of the request is not 
crucial in one embodiment of the invention. The request indicates to the merchant 
122 that the user of the set top box 152 desires to send a video/audio transmission. 

In response to the request, a receiving terminal of the merchant 122 
generates a notification, such as a text message or icon, for display to notify the 

20 merchant 122 of the upcoming video/audio transmission. Alternatively, the 
notification may take the form of an audio signal that is played on a speaker (not 
shown) of the receiving terminal of the merchant 122. 

If the receiving terminal of the merchant 122 is off-line or othenA/ise not 
available, the set top box 152 of the customer may wait until a timeout period has 

25 expired, after which it notifies the customer that the audio/video transmission cannot 
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be sent. Likewise, if the merchant 122 does not respond, or refuses to receive the 
transmission a "not-available" signal may be returned to the set top box 152. 

If the merchant 122 wishes to receive the video/audio transmission, 
the merchant 122 may press a suitable button on the receiving terminal, such as an 
5 "accept" button, which results in an acceptance signal being returned to the set top 
box 152. In one embodiment, the set top box 152 generates, in response to 
receiving the acceptance signal, a video or audio acceptance message to notify the 
customer that permission for the video/audio transmission has been granted. 

At a block 604, the set top box 152 and the receiving terminal of the 

10 merchant 122 may then initiate a handshake procedure to establish a 
communication protocol. In this case, the set top box 152 and the receiving terminal 
may negotiate a new protocol or reaffirm an existing protocol for video/audio 
communication. The appropriate protocol may need to be determined in one 
embodiment if the set top box 152 and the receiving terminal of the merchant 122 

15 have different video/audio conferencing capabilities. For example, the receiving 
terminal of the merchant 122 may be capable of video conferencing at a lower 
resolution (or frame rate), so the communication protocol is established as is 
suitable for this lower resolution (or frame rate). The communication protocol used 
may also depend on the bandwidth and/or reliability of the connection between the 

20 set top box 1 52 and the receiving terminal of the merchant 1 22. 

At this point, an active communication link is established between the 
set top box 152 and the receiving terminal of the merchant 122 across the network. 
In one embodiment, the customer then activates, at a block 606, a camera and/or 
microphone (not shown) by pressing, for example, a "cam" button on the remote 

25 control unit 158. In one implementation, the remote control unit 158 and/or the set 
top box 152 indicates at a block 608 activation of the camera by a visual 
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mechanism, such as an activity indicator (e.g., a light-emitting diode or LED). 
Thereafter at a block 610, the camera and/or microphone captures a video and/or 
audio signal (which is transmitted to the set top box 152). 

At a block 612, a converter within the set top box 152 then transforms 
5 the captured video/audio signal into a network-compatible video/audio stream for 
transmission over the network, such as over the cable network 134 and/or over the 
Internet 302. Thereafter at a block 614, the network-compatible video/audio stream 
is transmitted upstream to the network. As can be noted with reference to Figures 
1-3, the communication path for the upstream transmission may involve one or more 

10 head-ends, communication networks, and/or the Internet 302, using conventional 
routing techniques. 

In one embodiment, the network-compatible video/audio stream is 
then transmitted at a block 616 downstream from the network to the receiving 
terminal of the merchant 122. Thereafter, the network-compatible video/audio 

15 stream is transformed into a display-compatible video/audio signal at a block 618 for 
display on a television or display screen of the merchant 122 at a block 620. 

in a like manner, the receiving terminal of the merchant 122 may 
transmit video/audio information to the set top box 152 of the customer. Indeed, in 
one embodiment, multiple video/audio streams may be received and transmitted 

20 simultaneously by any particular terminal. Multiple video streams received by any 
particular terminal may be displayed on a display screen at the same time using 
picture-in-picture (PIP) techniques. Likewise, multiple audio streams may be mixed 
for playback on an appropriate terminal, in an embodiment MPEG PIDs may be 
used to allocate and identify requisite video streams. To provide for privacy, the 

25 customer may see the merchant 122 but not vice-versa in an embodiment. With this 
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privacy feature, the merchant 122 may listen to the customer via telephone or audio- 
conferencing, for example. 

Of course, the above-described technique depicted by the flow 
diagram 600 is only one possible technique for video and audio capture and 
5 communication within the scope of the Invention. In other embodiments, the set top 
box 152 may transmit a video/audio stream to the receiving terminal of the merchant 
122 (or vice versa) without waiting for an acceptance signal. The receiving terminal 
of the merchant 122 (or the set top box 152 of the customer) may record all 
incoming transmissions in a digital storage device. Thereafter, the merchant 122 

10 may review the stored video/audio streams and select which stream, if any, to 
display at a convenient time. This facilitates a form of "videomail." 

in yet another embodiment, the set top box 152 may be pre-configured 
to transmit video/audio information to the receiving terminal of the merchant 122, 
which has previously granted permission to receive the transmission. Accordingly, 

15 the user of the set top box 152 may simply press the "cam" button to immediately 
capture video/audio information and transmit the same to the receiving terminal for 
immediate display. 

The receiving terminal of the merchant 122 and/or the client terminal 
of the customer may comprise a PC or other device with a connection to the Internet 

20 302. Such other devices may include Internet appliances, PDAs, Internet-enabled 
cellular telephones, and the like. If these devices have varying videoconferencing 
capabilities, a handshaking procedure as described above can be used in 
determining a proper communication protocol. 

In conclusion, an embodiment of the invention allows an advertising 

25 service to present an interactive advertisement to a customer, such as via a 
television commercial. If the customer indicates an interest in the advertised 
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product by responding to the advertisement, then the advertising service or other 
mechanism detects a response to the interactive advertisement, identifies the 
customer, correlates the customer to a merchant, and then automatically connects 
the customer to the merchant (or triggers connection of the merchant to the 
5 customer). Thereafter, the merchant can fulfill the customer's response in a 
personalized and interactive manner. 

The above description of illustrated embodiments of the invention, 
including what is described in the Abstract, is not intended to be exhaustive or to 
limit the invention to the precise forms disclosed. While specific embodiments of, 

10 and examples for, the invention are described herein for illustrative purposes, 
various equivalent modifications are possible within the scope of the invention, as 
those skilled in the relevant art will recognize. 

For example, while various embodiments have been described above 
as involving viewing of the television commercial 402 and then processing the 

15 customer's response when the customer "clicks" the indicator 404, other types of 
interactive advertising and processing of customer responses are possible. For 
instance, the customer can receive an interactive advertisement via telephone, or 
the customer can respond to a visual interactive advertisement via telephone. With 
the use of voice recognition, speech processing, or other audio processing software, 

20 the customer's spoken response can be detected by the advertising service and 
then electronically processed (such as by identifying key words in the customer's 
response). Subsequently, the advertising service can perform additional correlation 
between the processed response and merchant information to identify a merchant 
that can service the customer, and then notify that merchant as to the origin and 

25 subject matter of the customer's response. 
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These modifications can be made to the invention in light of the above 
detailed description. The terms used in the following claims should not be 
construed to limit the invention to the specific embodiments disclosed in the 
specification and the claims. Rather, the scope of the invention is to be determined 
entirely by the following claims, which are to be construed in accordance with 
established doctrines of claim interpretation. 
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